Skip to content

ARROW-15454: [Python] Try to make CSV cancellation test more robust#12260

Closed
pitrou wants to merge 1 commit into
apache:masterfrom
pitrou:ARROW-15454-csv-cancellation
Closed

ARROW-15454: [Python] Try to make CSV cancellation test more robust#12260
pitrou wants to merge 1 commit into
apache:masterfrom
pitrou:ARROW-15454-csv-cancellation

Conversation

@pitrou

@pitrou pitrou commented Jan 25, 2022

Copy link
Copy Markdown
Member

Use signal.set_wakeup_fd to wait reliably for the signal to get to the Python interpreter.

@github-actions

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@pitrou

pitrou commented Jan 25, 2022

Copy link
Copy Markdown
Member Author

@github-actions crossbow submit -g python

@github-actions

Copy link
Copy Markdown

Revision: 12f1b96519fe388e81ca400c075b900b705617aa

Submitted crossbow builds: ursacomputing/crossbow @ actions-1493

Task Status
test-conda-python-3.10 Github Actions
test-conda-python-3.7 Github Actions
test-conda-python-3.7-hdfs-2.9.2 Github Actions
test-conda-python-3.7-hdfs-3.2.1 Github Actions
test-conda-python-3.7-kartothek-latest Github Actions
test-conda-python-3.7-kartothek-master Github Actions
test-conda-python-3.7-pandas-0.24 Github Actions
test-conda-python-3.7-pandas-latest Github Actions
test-conda-python-3.7-spark-v3.1.2 Github Actions
test-conda-python-3.8 Github Actions
test-conda-python-3.8-hypothesis Github Actions
test-conda-python-3.8-pandas-latest Github Actions
test-conda-python-3.8-pandas-nightly Github Actions
test-conda-python-3.8-spark-v3.2.0 Github Actions
test-conda-python-3.9 Github Actions
test-conda-python-3.9-dask-latest Github Actions
test-conda-python-3.9-dask-master Github Actions
test-conda-python-3.9-pandas-master Github Actions
test-conda-python-3.9-spark-master Github Actions
test-debian-11-python-3 Azure
test-fedora-33-python-3 Azure
test-ubuntu-20.04-python-3 Azure

Comment thread appveyor.yml Outdated

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This allows to run the cancellation tests on Windows.

@pitrou pitrou requested a review from lidavidm January 25, 2022 20:17
@pitrou

pitrou commented Jan 25, 2022

Copy link
Copy Markdown
Member Author

@kszucs Perhaps this can allow us to remove the workarounds added in #12243

@kszucs

kszucs commented Jan 25, 2022

Copy link
Copy Markdown
Member

Since it only occurred in the M1 wheel verification tests I think it's fine to keep the workaround for 7.0 and ship this with 8.0.

@lidavidm lidavidm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one question.

Comment thread python/pyarrow/tests/util.py Outdated
@kszucs

kszucs commented Jan 25, 2022

Copy link
Copy Markdown
Member

@github-actions crossbow submit wheel-macos-big-sur-cp38-arm64

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now the failure should occur unless the patch resolves it.

@github-actions

Copy link
Copy Markdown

Revision: 9525671caa3d0bb5b2913733253783c1fe1525d2

Submitted crossbow builds: ursacomputing/crossbow @ actions-1495

Task Status
wheel-macos-big-sur-cp38-arm64 Github Actions

@kszucs

kszucs commented Jan 25, 2022

Copy link
Copy Markdown
Member

@github-actions crossbow submit wheel-macos-big-sur-cp38-arm64

@github-actions

Copy link
Copy Markdown

Revision: 9525671caa3d0bb5b2913733253783c1fe1525d2

Submitted crossbow builds: ursacomputing/crossbow @ actions-1496

Task Status
wheel-macos-big-sur-cp38-arm64 Github Actions

@kszucs kszucs force-pushed the ARROW-15454-csv-cancellation branch from 9525671 to 3490d79 Compare January 25, 2022 21:08
@pitrou

pitrou commented Jan 26, 2022

Copy link
Copy Markdown
Member Author

Looks like the changes here weren't enough to fix the test on macOS M1.

Use `signal.set_wakeup_fd` to wait reliably for the signal to get to the Python interpreter.
@pitrou pitrou force-pushed the ARROW-15454-csv-cancellation branch from 3490d79 to 3b324e1 Compare January 26, 2022 14:59
@pitrou

pitrou commented Jan 26, 2022

Copy link
Copy Markdown
Member Author

I've removed the Crossbow tasks changes and will merge if green.

@pitrou

pitrou commented Jan 26, 2022

Copy link
Copy Markdown
Member Author

Note the Python MacOS build was cancelled because the dependency install phase took too long.

@pitrou pitrou closed this in 9e69ccf Jan 26, 2022
@pitrou pitrou deleted the ARROW-15454-csv-cancellation branch January 26, 2022 17:47
@ursabot

ursabot commented Jan 26, 2022

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = f9f6fdb and contender = 9e69ccf. 9e69ccf is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️2.03% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.3% ⬆️0.04%] ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants